Skip to content

fix: Clarify error message for podman restore without --tcp-established#29026

Open
simek-m wants to merge 1 commit into
podman-container-tools:mainfrom
simek-m:fix/RHEL-186005-restore-tcp-established
Open

fix: Clarify error message for podman restore without --tcp-established#29026
simek-m wants to merge 1 commit into
podman-container-tools:mainfrom
simek-m:fix/RHEL-186005-restore-tcp-established

Conversation

@simek-m

@simek-m simek-m commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Newer versions of crun use show_criu_log() to report errors from CRIU and output multiple
error lines. It lead to a confusing behavior in readConmonPipeData() in Podman, because
the JSON Unmarshal() failed to parse multiple JSON objects over multiple lines. When podman restore was used without --tcp-established or --tcp-close, a confusing error message:

Error: crun: (00.022418) Error (criu/cgroup.c:1970): cg: cgroupd: recv req error: No such file or directory: OCI runtime attempted to invoke a command that was not found

was output, instead of the former:

crun: CRIU restoring failed -52.

In this PR, I parse all errors from the log and try to match
"Connected TCP socket in image". If found,
a clear error message is returned:

checkpoint contains established TCP connections, restore requires --tcp-established or --tcp-close: OCI runtime error

For other error messages, the existing behavior is retained. Tests were updated with the changed error message.

Steps to reproduce:

podman run -d --name server quay.io/libpod/testimage:20221018 nc -lp 8888
serverIP=`podman inspect server --format {{.NetworkSettings.IPAddress}}`
podman run -d -i --name client quay.io/libpod/testimage:20221018 nc $serverIP 8888
podman container checkpoint --tcp-established server
podman container restore server

Fixes: https://redhat.atlassian.net/browse/RHEL-186005

Checklist

Ensure you have completed the following checklist for your pull request to be reviewed:

  • Certify you wrote the patch or otherwise have the right to pass it on as an open-source patch by signing all
    commits. (git commit -s). (If needed, use git commit -s --amend). The author email must match
    the sign-off email address. See CONTRIBUTING.md
    for more information.
  • Referenced issues using Fixes: #00000 in commit message (if applicable)
  • Tests have been added/updated (or no tests are needed)
  • Documentation has been updated (or no documentation changes are needed)
  • All commits pass make validatepr (format/lint checks)
  • Release note entered in the section below (or None if no user-facing changes)

Does this PR introduce a user-facing change?

If `podman restore` is used without with a checkpoint containing established TCP connections and without the flags --tcp-established or --tcp-close, the error message "checkpoint contains established TCP connections, restore requires --tcp-established or --tcp-close: OCI runtime error" is returned.

Newer version of crun use show_criu_log() to
report errors from CRIU and output multiple
error lines. It lead to confusing behavior
in readConmonPipeData() in Podman, because
the JSON Unmarshal() failed to parse multiple
JSON objects over multiple lines. When
podman restore was used without --tcp-established,
a confusing error message:
  cgroupd: recv req error: No such file or directory
was output, instead of the former:
  crun: CRIU restoring failed -52.

Parse all errors from the log and try to match
"Connected TCP socket in image". If found,
return a clear error message. Update tests.

Fixes: https://redhat.atlassian.net/browse/RHEL-186005
Signed-off-by: Marek Simek <msimek@redhat.com>
@mheon

mheon commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Do these work with both runc and crun?

Comment thread libpod/oci_util.go
if strings.Contains(e.Msg, "Connected TCP socket in image") {
return fmt.Errorf("checkpoint contains established TCP connections, restore requires --tcp-established or --tcp-close: %w", define.ErrOCIRuntime)
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that does not seem maintainable, we should not make assumptions about the error text from crun or criu, it changes often enough. Sure the tests depend on it to some extend but we should not make the code depend, i.e. this may not work for another runtime.

It would be better to work with crun and criu to produce better errors to begin with.

@simek-m

simek-m commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

@mheon

Do these work with both runc and crun?

Yes, it should work the same with both.

[root@lima-default]~# podman container restore server
Error: OCI runtime error: checkpoint contains established TCP connections, restore requires --tcp-established or --tcp-close

[root@lima-default]~# podman info --format '{{.Host.OCIRuntime.Name}}'
runc

It's a CRIU error being matched.

@Luap99

that does not seem maintainable, we should not make assumptions about the error text from crun or criu, it changes often enough. Sure the tests depend on it to some extend but we should not make the code depend, i.e. this may not work for another runtime.

It would be better to work with crun and criu to produce better errors to begin with.

I agree, but that's exactly what caused this bug report.
Not only the tests depend on the error output, but there's getOCIRuntimeError() that tries to match the error text and return a corresponding Podman error based on that. Because of changes in the crun output, the cg: cgroupd: recv req error: No such file or directory error was incorrectly converted to OCI runtime attempted to invoke a command that was not found by this regex and the fact that other error lines from CRIU were swallowed by the buggy JSON Unmarshal not handling multiple objects:

if match := regexp.MustCompile("(?i).*executable file not found in.*|.*no such file or directory.*|.*open executable.*").FindString(runtimeMsg); match != "" {
	errStr := match
	if includeFullOutput {
		errStr = runtimeMsg
	}
	return fmt.Errorf("%s: %s: %w", name, strings.Trim(errStr, "\n"), define.ErrOCIRuntimeNotFound)
}

Unfortunately, there's AFAIK no specific error code returned by crun or CRIU in this case that could be used and that might not be stable either.

Alternatively, the whole error output from crun could be shown to the user, or there can be no change at all - it still works as documented.

But in my opinion and from the perspective of a user, I'd like to see a helpful error message in this case, even though it's documented in https://docs.podman.io/en/latest/markdown/podman-container-restore.1.html#tcp-established

@Luap99

Luap99 commented Jul 2, 2026

Copy link
Copy Markdown
Member

Not only the tests depend on the error output, but there's getOCIRuntimeError() that tries to match the error text and return a corresponding Podman

which I absolutely hate, the fact we eat up errors and convert them often also creates just more confusing errors as well. It also often has no way of working with multiple different runtimes.

Alternatively, the whole error output from crun could be shown to the user, or there can be no change at all - it still works as documented.

I think parsing the full error log is the right thing to do, if it gives us many lines we should forward all of them

@simek-m

simek-m commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@Luap99

Not only the tests depend on the error output, but there's getOCIRuntimeError() that tries to match the error text and return a corresponding Podman

which I absolutely hate, the fact we eat up errors and convert them often also creates just more confusing errors as well. It also often has no way of working with multiple different runtimes.

Alternatively, the whole error output from crun could be shown to the user, or there can be no change at all - it still works as documented.

I think parsing the full error log is the right thing to do, if it gives us many lines we should forward all of them

Thank you. Do you mean getting rid of the whole getOCIRuntimeError mechanism and just forwarding the received error output with some generic error? Because I'm afraid it's not possible to limit the change to just the case in this bug without relying on the crun/CRIU error message text.

If so, I think it's quite a big and possibly breaking change in the error messages that would users receive.

Anyway, I'd still like finding a way of providing some context to users (in this case guiding them to use the --tcp-established or --tcp-close

@Luap99

Luap99 commented Jul 3, 2026

Copy link
Copy Markdown
Member

Thank you. Do you mean getting rid of the whole getOCIRuntimeError mechanism and just forwarding the received error output with some generic error? Because I'm afraid it's not possible to limit the change to just the case in this bug without relying on the crun/CRIU error message text.

I would love to go into that direction.

At the minimum I think we should always forward all error messages, i.e. just use the parseOCIErrors() for all messages. We can return a multi line error with errors.Join() for all of them.

If so, I think it's quite a big and possibly breaking change in the error messages that would users receive.

Well error messages should provide the proper context what went wrong, they are not a stable API by design.
There is some special notes here with the 126/127 exit codes we provide for certain errors.

Anyway, I'd still like finding a way of providing some context to users (in this case guiding them to use the --tcp-established or --tcp-close

I don't strictly disagree but overall I find most of these assumption just do not hold long term.
Connected TCP socket in image is a hard coded assumption about a error string from criu, they can just that or maybe the error test could mean something else.

If we want better errors I would argue this needs to be pushed back to criu. They know whenever the tcp-established option is set or not, so they could just as well change the message to Connected TCP socket in image but neither the closing nor preserving of tcp sockets was requested which would help a lot already.

Also the other problem with this I did not even touch yet is that we really should not hard code cli flags in error message in the backend code. We have a fully support REST API, returning cli arguments to some user who is not using the cli is not useful either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants